Failure to Disrupt: Why Technology Alone Can’t Transform Education
Authors: Justin Reich
 (Z-Library)_thumbnail.jpg){kind=spacer}
Overview
In “Failure to Disrupt,” I explore why technology alone hasn’t transformed education and how we can move toward a more equitable and effective future for learning at scale. I argue that despite decades of hype and optimistic predictions, new technologies have primarily served as modest supplements to traditional educational practices rather than as disruptive forces for change. I identify four key challenges, or “as-yet intractable dilemmas,” that explain why this is the case: the curse of the familiar, the edtech Matthew effect, the trap of routine assessment, and the toxic power of data and experiments.
Throughout the book, I explore a range of large-scale learning environments, from MOOCs to adaptive tutors to peer-guided learning communities, examining their history, pedagogical approaches, and impact on learners. I argue that while new technologies have opened up important possibilities for improving learning, they are often adopted in ways that reinforce existing inequalities and fail to address the fundamental complexities of teaching and learning.
“Failure to Disrupt” is written for educators, policymakers, technology developers, and anyone interested in the future of learning. I offer a framework for understanding the challenges and opportunities of learning at scale and present a set of design principles for digital equity that can help guide future innovation and investment. I argue that rather than seeking radical transformations driven by technology, we should focus on building communities of educators dedicated to progressive pedagogical change and embrace a long-term commitment to tinkering and continuous improvement.
Book Outline
1. Introduction
New technologies rarely lead to radical shifts in how we teach and learn. Schools and colleges are inherently conservative institutions, more likely to adapt technologies to existing practices than to be transformed by them. The pandemic has revealed this conservative nature even more clearly, as educators have largely sought to replicate existing classroom experiences in online environments.
Key concept: Schools and colleges are among the most durable and conservative of our social institutions. They prepare people for the future by connecting them with knowledge and wisdom from the past.
2. Three Genres of Learning at Scale
It’s helpful to classify large-scale learning technologies into three genres: instructor-guided like MOOCs, algorithm-guided like adaptive tutors, and peer-guided like networked learning communities. Each genre has a distinct history, pedagogical approach, and set of successes and failures in formal education. This framework helps us analyze how a new technology might contribute to existing educational systems.
Key concept: Large-scale learning environments can be classified into three genres: instructor-guided, algorithm-guided, and peer-guided.
3. Instructor-Guided Learning at Scale: Massive Open Online Courses
MOOCs have not lived up to the hype of disrupting higher education, expanding access, or generating groundbreaking research insights about learning. While millions have registered for MOOCs, they disproportionately benefit already educated, affluent learners. MOOCs are now primarily used as supplements to existing educational infrastructure, offering professional master’s degrees, executive education programs, and single course certificates.
Key concept: MOOCs are good for helping people pursue a second or third degree.
4. Algorithm-Guided Learning at Scale: Adaptive Tutors and Computer-Assisted Instruction
Adaptive tutors, powered by algorithms and automated assessment, have shown promise in mathematics and early reading instruction, particularly as a supplement to classroom teaching. However, these technologies struggle to assess complex human performances like reasoning, argumentation, and creativity, limiting their applicability across the curriculum. The challenge for developers is to expand the range of learning that can be computationally assessed.
Key concept: The trap of routine assessment is that computers can only assess what computers themselves can do, so that’s what we teach students.
5. Peer-Guided Learning at Scale: Networked Learning Communities
While less common in formal education, peer-guided learning environments like connectivist MOOCs and the Scratch programming community demonstrate the power of online networks for supporting interest-driven learning. In these environments, learners set their own pace, explore their passions, and connect with peers to develop new skills and knowledge. However, these approaches require a high degree of learner self-direction and can be challenging to integrate into traditional schooling structures.
Key concept: Content is a MacGuffin.
6. Testing the Genres of Learning at Scale: Learning Games
Learning games, like other large-scale learning technologies, can be classified into the three genres. Many learning games, like Math Blaster, rely on gamification — adding game-like elements to traditional drill-and-practice activities — and ultimately function as instructor-guided experiences. These approaches offer modest benefits but do not fundamentally change how learning happens.
Key concept: Chocolate-covered broccoli.
7. The Curse of the Familiar
New technologies are far more likely to be adapted to existing practices in schools than to lead to substantial change. The curse of the familiar describes this tendency: technologies that look like familiar classroom routines are easy to adopt but less likely to lead to innovation. Conversely, genuinely novel technologies often confuse or overwhelm users and are difficult to integrate into existing systems.
Key concept: The curse of the familiar is that easily adopted technologies will be those that replicate existing classroom practices, but digitizing what teachers and students already do is unlikely to lead to substantial improvements in schools.
8. The Edtech Matthew Effect
Despite the promise of democratizing learning, new technologies often exacerbate existing inequalities. The edtech Matthew effect describes how the benefits of new technologies, even free and open ones, accrue disproportionately to learners who are already affluent and well connected. Financial, technical, social, and cultural barriers all contribute to these patterns of uneven access and usage.
Key concept: New resources—even free, online resources—are more likely to benefit already affluent learners with access to networked technology and access to networks of people who know how to take advantage of free online resources.
9. The Trap of Routine Assessment
Automated assessment systems, while improving, are fundamentally limited by the trap of routine assessment. These systems excel at evaluating routine tasks that are easily structured and graded, but they struggle with assessing complex communication skills, unstructured problem solving, and creativity – the very skills that are increasingly valued in the 21st century. As a result, our education system often focuses on what can be easily assessed at scale, even when those skills are not the ones most essential for learners’ future success.
Key concept: The trap of routine assessment is that computers can only assess what computers themselves can do, so that’s what we teach students.
10. The Toxic Power of Data and Experiments
Large-scale learning environments generate vast amounts of data about learners, offering unprecedented opportunities for research and continuous improvement. However, these data also pose significant privacy risks. The toxic power of data and experiments highlights the need for carefully weighing the potential benefits of research with the dangers of data breaches, surveillance, and the inappropriate quantification of human performance. Educators, policymakers, and technology developers must work together to establish ethical guidelines for data collection, storage, and usage.
Key concept: Toxic assets, like radioactive materials—which can both save lives and cause cancer—must be used with great care.
11. Conclusion
Meaningful change in education will not come from charismatic technologists or technology alone. Instead, it will require the collaborative efforts of educators, researchers, designers, and policymakers working to improve teaching and learning in incremental ways. Building communities of educators dedicated to progressive pedagogical change is essential for realizing the potential of learning technologies and creating a more equitable future for education.
Key concept: Change won’t come from heroic developers or even technology firms, but from communities of educators, researchers, and designers oriented toward innovative pedagogy and a commitment to educational equity. We need villages, not heroes.
Essential Questions
1. Why haven’t technologies transformed education as predicted?
The hype surrounding educational technologies often promises radical transformations of teaching and learning, but rarely delivers. The reality is that schools are complex systems with deeply entrenched practices and structures. New technologies are more likely to be adapted to fit within these existing systems than to disrupt them. This conservative nature is driven by a combination of factors, including teacher workload, anxiety about new approaches, the need to maintain order and control, and the pressure of high-stakes testing.
2. What are the different genres of learning at scale and how do they interact with formal education systems?
Learning at scale can be classified into three distinct genres: instructor-guided, algorithm-guided, and peer-guided. Understanding these genres and their historical trajectories helps to make predictions about how new technologies will be adopted and used in educational contexts. For example, MOOCs, as instructor-guided experiences, are likely to continue serving the needs of already educated professionals, while adaptive tutors, as algorithm-guided experiences, may prove valuable as supplements to math instruction, but are unlikely to transform whole-school curricula.
3. What is the ‘curse of the familiar’ and how can it be overcome?
The ‘curse of the familiar’ describes the tendency for easily adopted technologies to be those that closely resemble existing classroom practices. This can lead to modest efficiency gains, but is unlikely to result in substantial improvements to learning. Conversely, genuinely novel technologies often confuse users or require significant changes to existing practices, making widespread adoption difficult. Breaking the curse of the familiar requires building communities of educators who are committed to progressive pedagogical change and are willing to experiment with new approaches.
4. What is the edtech Matthew effect and how can we design for greater equity?
The edtech Matthew effect describes the phenomenon whereby new technologies, even those offered for free, disproportionately benefit already advantaged learners. This is due to a combination of factors, including access to technology and broadband, supportive home environments, and familiarity with navigating online learning resources. Addressing this inequality requires moving beyond simply expanding access to technology and instead focusing on building the capacity of learners, families, and communities to engage meaningfully with new learning opportunities.
5. What is the ‘trap of routine assessment’ and how does it limit the potential of learning at scale?
The ability to computationally assess learning at scale is fundamentally limited by the ‘trap of routine assessment’. Current assessment systems excel at evaluating routine tasks with clear right or wrong answers, like those found in math or computer programming. However, they struggle to assess complex communication skills, unstructured problem solving, and creativity - the skills that are increasingly valued in the 21st century. Overcoming this limitation requires ongoing research and development in new assessment technologies, such as stealth assessment, that can evaluate a wider range of human performances.
Key Takeaways
1. The focus on computationally assessable skills limits the scope of learning.
The “trap of routine assessment” highlights the limitations of current automated assessment systems. While these systems excel at evaluating routine tasks with clear right or wrong answers, they struggle to assess complex, higher-order thinking skills. This means that if we want to teach and assess skills like problem-solving, creativity, and critical thinking, we need to move beyond simply digitizing existing classroom practices and instead develop new approaches to both teaching and assessment.
Practical Application:
A team designing a new AI-powered math tutoring system could use this takeaway to inform their development and implementation strategy. Rather than simply replicating existing drill-and-practice exercises, they could focus on creating activities that encourage students to engage in mathematical modeling, problem framing, and explanation of reasoning. They could also invest in teacher training and support to help educators integrate the tool in ways that promote deeper learning.
2. Social and cultural factors are critical for achieving equity in edtech.
Technology alone cannot address the social and cultural factors that contribute to educational inequality. Students from disadvantaged backgrounds often face a variety of obstacles to accessing and engaging meaningfully with new learning opportunities, including lack of access to technology and broadband at home, limited exposure to adults with technical expertise, and feelings of social identity threat. Designing for equity requires attending to these complex social and cultural contexts.
Practical Application:
When developing a chatbot for high school students exploring college and career options, designers can integrate this takeaway by including features that allow students to connect with mentors and peers from diverse backgrounds. The chatbot could also highlight pathways that challenge traditional stereotypes and encourage students to consider a wider range of possibilities based on their interests, rather than just their academic performance.
3. Community is essential for driving meaningful change in education.
Technology adoption in education is more successful when it is driven by community, not just by individual teachers or administrators. Building strong communities of educators who are invested in pedagogical change and who see technology as a tool for supporting that change is essential for moving beyond simply replicating existing practices with new tools. This requires a long-term commitment to professional development, relationship building, and shared vision.
Practical Application:
An edtech company creating a new online learning platform for professional development could apply this takeaway by engaging teachers in the design process from the outset. They could conduct user research to understand teachers’ needs, preferences, and existing practices, and then use that feedback to develop a platform that feels relevant and useful to educators. They could also prioritize building community features that allow teachers to connect with peers, share ideas, and support one another’s learning.
Suggested Deep Dive
Chapter: The Edtech Matthew Effect
This chapter provides a compelling analysis of how new technologies often exacerbate existing inequalities and offers concrete examples of how to design for greater equity in edtech.
Memorable Quotes
Introduction. 8
The path of educational progress more closely resembles the flight of a butterfly than the flight of a bullet.
Algorithm-Guided Learning at Scale. 56
New learning technologies rarely innovate on these fundamental pedagogical ideas. Instead, they reenact them.
Peer-Guided Learning at Scale. 76
Content is a MacGuffin.
Testing the Genres of Learning at Scale. 101
Chocolate-covered broccoli.
The Edtech Matthew Effect. 132
For whoever has will be given more, and they will have an abundance. Whoever does not have, even what they have will be taken away from them.
Comparative Analysis
While “Failure to Disrupt” echoes anxieties about techno-solutionism found in works like Audrey Watters’ “Teaching Machines” and Neil Selwyn’s “Distrusting Educational Technology,” Reich’s book distinguishes itself through its focus on the structural constraints of educational systems. While others critique the hype and ideology surrounding edtech, Reich provides a nuanced framework for understanding how existing incentives and organizational structures shape technology adoption and impact. Unlike the sweeping indictments of technology often found in critical pedagogy, Reich embraces a ‘tinkerer’s’ perspective, acknowledging both the limitations and potential benefits of learning technologies. His emphasis on community, incremental change, and research-driven design provides a valuable counterpoint to the utopian visions of charismatic technologists.
Reflection
Reich’s “Failure to Disrupt” offers a sobering yet hopeful perspective on the role of technology in education. While his critiques of techno-solutionism and the hype surrounding educational technologies are well-founded, his focus on incremental change might be overly cautious. Perhaps the ‘butterfly’ analogy underestimates the potential for technology to accelerate certain aspects of learning, particularly when combined with systemic reforms. While acknowledging the edtech Matthew effect, Reich might give insufficient attention to the ways in which technologies can be designed to mitigate bias and promote equity. His emphasis on building communities of educators, while important, could be misconstrued as downplaying the role of individual agency and innovation. Despite these potential limitations, “Failure to Disrupt” provides a valuable framework for analyzing the complexities of learning at scale and offers practical guidance for navigating the challenges and opportunities of a rapidly evolving educational landscape. It encourages us to move beyond simplistic narratives of disruption and instead embrace a long-term commitment to research, collaboration, and continuous improvement.